In recent years, we have greeted the age of multimedia in which audio, video and other data are integrally handled, and the conventional information media (i.e., means for transmitting information media), such as newspapers, magazines, radios, and telephones, have been adopted as the subjects of multimedia. Generally, "multimedia" means to represent, not only characters, but also diagrams, speeches, and especially images simultaneously in relation with each other. In order to handle the conventional information media as the subjects of multimedia, it is necessary to transform the information into a digital format.
When the quantity of data possessed by each information medium described above is estimated as the quantity of digital data, in case of characters, the data quantity for each character is 1.about.2 byte. However, in case of speech, the data quantity is 64 Kbits per second (quality for telecommunication) and, in case of moving picture, it is more than 100 Mbits per second (quality for current television broadcasting). So, as for the information media described above, it is not practical to handle such massive data as it is in the digital format. For example, although visual telephones have already been put to practical use by ISDN (Integrated Services Digital Network) having a transmission rate of 64 Kbps-1.5 Mbps, it is impossible to transmit an image of a television camera as it is by the ISDN.
So, data compression technologies are demanded. In case of visual telephones, the moving picture compression technologies standardized as H.261 and H.263 by ITU-T (International Telecommunication Union-Telecommunication Sector) are employed. Further, according to the data compression technology based on MPEG1, it is possible to record image data, together with audio data, in an ordinary music CD (compact disk).
MPEG (Moving Picture Experts Group) is an international standard of data compression for data of a moving picture (an image signal of a moving picture). In MPEG1, data of a moving picture is compressed to 1.5 Mbps, i.e., data of a television signal is compressed to about 1/100. Since the transmission rate to which MPEG1 is directed is limited to about 1.5 Mbps, MPEG2 is standardized to meet the demand for higher image quality. In MPEG2, data of a moving picture is compressed to 2.about.15 Mbps.
Under the existing circumstances, standardization of MPEG4 is now proceeded by the working group for standardization of MPEG1 and MPEG2 (ISO/IEC JTC1/SC29/WG11). MPEG4 enables coding and signal operation in object units, and thereby realizes new functions required in the age of multimedia. MPEG4 has originally aimed at standardization of a coding method at a low bit rate, but the aim of standardization is now extended to a more versatile coding process at a high bit rate adaptable to an interlaced image.
FIGS. 8(a)-8(f) are diagrams for explaining an object-by-object (object-based) coding process.
Shown in FIG. 8(a) is an image space Ts obtained from a color signal comprising a luminance signal and a chrominance signal and including an object To corresponding to an image. Shown in FIG. 8(b) is an image space Ss obtained from a shape signal indicating a shape of the object. In the image space Ss shown in FIG. 8(b), a portion painted in black is an inside region of an object So. The fish image (object) To in the image space Ts is represented by a color signal corresponding to pixels composing the inside region (pixels inside the object).
According to MPEG4, an image signal comprising the shape signal and the color signal corresponding to an object is coded for each unit region (macroblock) composed of a predetermined number of pixels. While MPEG1 and MPEG2 have introduced a technique for coding the color signal for each macroblock, MPEG4 adopts a technique for coding the color signal and the shape signal for each macroblock.
FIG. 8(c) schematically shows a blocking process in which the color signal is divided into signals corresponding to respective macroblocks Tmb composing the image space Ts. FIG. 8(d) schematically shows a blocking process in which the shape signal is divided into signals corresponding to respective macroblocks Smb composing the image space Ss.
With a view to performing an efficient coding process for a frame (image space)including an object, according to MPEG4, only the inside of a rectangular region including the object in the frame rather than an image signal corresponding to the entire frame is coded. The rectangular region is called a "bounding box".
To be specific, for the color signal, as shown in FIG. 8(e), the bounding box Tb is divided into portions corresponding to respective macroblocks (blocking process). Also, for the shape signal, as shown in FIG. 8(f), a bounding box Sb is divided into portions corresponding to macroblocks (blocking process).
The image signal comprising the color signal and the shape signal is subjected to a motion compensation coding process for each macroblock like a motion compensation coding process according to MPEG1 and MPEG2.
In this motion compensation coding process, there should be a match of motion change from frame to frame between the color and shape signals. In actuality, however, there is no match between real motion of the object and motion of the object of respective signals which improves coding efficiency.
The motion compensation coding process is to perform coding to a motion compensation error of the color or shape signal. If motion information giving the motion compensation error smaller than that of motion information (motion vector) indicating real motion of the object is obtained in the motion compensation coding process, it is more desirable to perform the motion compensation process based on the motion information giving the smaller error, because this increases coding efficiency. Since the color signal is a multi-valued signal, and the shape signal is a binary signal for distinguishing between pixels inside the object and pixels outside the object, their qualities differ from each other a little, and correspondingly the motion vectors thereof giving smaller errors differ from each other a little.
The motion compensation error is a difference value between a color or shape signal corresponding to a macroblock to-be-processed (target macroblock)i.e., macroblock to be coded, in a current frame (frame being coded) and a color or shape signal corresponding to a region equal to the macroblock to-be-processed in size (prediction macroblock) in a previous (past) frame (coded frame serving as a reference frame). The motion vector of each macroblock is position information indicating the positioning relationship between the macroblock to-be-processed on the frame and the prediction macroblock.
According to MPEG4, the color signal and the shape signal are subjected to the motion compensation coding process based on the motion vectors (hereinafter referred to as a color motion vector and a shape motion vector) obtained from the corresponding color and shape signals, respectively.
Although a match might not be always found between the color motion vector and the corresponding shape motion vector with respect to size or direction, there is high correlation between them. For this reason, in MPEG4, according to the shape motion vector of the coded macroblock in a current frame and the corresponding color motion vector, a prediction value of the shape motion vector of the target macroblock is generated, and a difference value between the prediction value and the value of the shape motion vector of the target macroblock, i.e., information of the shape motion vector of the target macroblock is coded.
A description will be given of a prior art image decoding apparatus according to MPEG4.
FIG. 9 is a block diagram for explaining an MPEG compliant image coding apparatus according to the prior art.
Turning now to FIG. 9, there is shown an image coding apparatus 1000. The image coding apparatus 1000 comprises a color coding section 1100 which subjects the color signal to a coding process including the motion compensation coding process and a shape coding section 1200 which subjects the shape signal to a coding process including the motion compensation coding process.
The shape coding section 1200 includes a blocking unit 1210 which performs a blocking process in such a way that a shape signal Ss corresponding to an object input through an input terminal 1201 is divided into signals corresponding to macroblocks and outputs blocked shape signals Bs, and a memory 1260 which stores a shape signal (locally decoded shape signal) Lds of a coded frame (previous frame) as a reference shape signal MLds.
The shape coding section 1200 further includes a shape motion detector 1240 which detects a motion vector (shape motion vector) MVs of the target macroblock to-be-processed based on the reference shape signal MLds stored in the memory 1260 and the blocked shape signal Bs output from the blocking unit 1210, and outputs a motion compensation error signal Dfs of the shape signal obtained in this detection process, a shape motion compensator 1250 which performs the motion compensation process to the blocked shape signal Bs based on the reference shape signal MLds and the shape motion vector MVs and outputs a prediction shape signal Ps for the target macroblock (shape signal of the prediction macroblock), and a mode decision unit 1280 which performs a coding mode decision process for the shape signal based on the blocked shape signal Bs output from the blocking unit 1210 and the motion compensation error signal Dfs output from the shape motion detector 1240 and outputs a shape coding mode signal Mo.
The shape coding mode signal Mos indicates that the target macroblock corresponding to the shape signal is a macroblock outside the object, an intra-macroblock which has been intra-frame coded using correlation between pixel values in a frame, or an inter-macroblock which has been inter-frame coded using correlation of pixel values between frames. The macroblock outside the object is a macroblock, the pixels of which are all positioned outside the object. The intra-macroblock and the inter-macroblock are macroblocks inside the object, and at least some of their pixels are positioned inside the object.
The shape coding section 1200 still further includes a shape encoder 1220 which performs an arithmetic coding process (inter-frame coding process) with reference to the prediction shape signal Ps or an arithmetic coding process (intra-frame coding process) without reference to the prediction shape signal Ps for the blocked shape signal Bs according to the shape coding mode signal Mos and outputs a coded shape signal Cs, and outputs a decoded signal (locally decoded shape signal) Lds of the coded shape signal Cs, a variable length encoder 1230 which performs a variable length coding process to the coded shape signal Cs and the shape coding mode signal Mos and outputs a coded shape signal Es, and a motion vector coding unit 1270 which performs coding to the shape motion vector MVs according to the coding mode signal Mos and information Mmvt and Mmot relating to the motion vector sent from the color coding apparatus 1100 and outputs a coded motion vector signal Emvs.
In this shape coding apparatus 1200, the locally decoded shape signal Lds from the shape encoder 1220 is stored in the memory 1260 as the reference shape signal MLds, which is output to the shape motion detector 1240 and the shape motion compensator 1250 and used in the motion detection process and the motion compensation process for the shape signal, respectively. The locally decoded shape signal Lds is sent to the color coding section 1100 and used in the motion detection process and the motion compensation process for the color signal.
The color coding section 1100 includes a blocking unit 1100 which performs a blocking process in such a way that a color signal St corresponding to an object input through an input terminal 1101 is divided into signals corresponding to respective macroblocks and outputs blocked color signals Bt and a memory 1160 which stores a color signal (locally decoded color signal) Ldt of a coded frame (previous frame) as a reference color signal MLdt.
The color coding section 1100 further includes a color motion detector 1140 which detects a color motion vector MVt of the target macroblock based on the reference color signal MLdt stored in the memory 1160, the locally decoded shape signal Lds from the shape coding section 1200, and the blocked color signal Bt from the blocking unit 1100, and outputs a motion compensation error signal DFt of the color signal, a mode decision unit 1180 which performs a coding mode decision process for the color signal based on the blocked color signal Bt from the blocking unit 1110, the motion compensation error signal DFt from the color motion detector 1140, and the blocked shape signal BS from the shape coding section 1200 and outputs a color coding mode signal Mot, and a color motion compensator 1150 which performs a motion compensation process to the blocked color signal Bt based on the reference color signal MLdt, the color motion vector MVt, and the locally decoded shape signal Lds from the shape coding section 1200, and outputs a prediction color signal Pt of the target macroblock (color signal of the prediction macroblock).
The color coding mode signal Mot indicates the target macroblock is the macroblock outside the object, the intra-macroblock, or the inter-macroblock.
The color coding section 1100 still further includes a color encoder 1120 which performs a wave coding process with reference to the prediction color signal Pt(inter-frame coding process) or a wave coding process without reference to the prediction color signal Pt (intra-frame coding process) for the blocked color signal Bt and outputs a coded color signal Ct and outputs a decoded signal (locally decoded color signal) Ldt of the coded color signal Ct, a variable length encoder 1130 which performs variable length coding to the coded color signal Ct and the color coding mode signal Mot and outputs a coded color signal Et, a motion vector coding unit 1170 which perform coding to the color motion vector MVt and outputs a coded motion vector signal Emvt and outputs the information Mmvt and Mmot relating to the color motion vector to the motion vector coding unit 1270.
In this color coding section 1100, the locally decoded color signal Lds from the color encoder 1120 is stored in the memory 1160 as the reference color signal MLds, which is output to the color motion detector 1140 and the color motion compensator 1150 and used in motion detection and motion compensation processes for the color signal, respectively.
It should be remembered that the image signal according to MPEG includes a transparency signal indicating the composition ratio in which the color signal corresponding to the object and another color signal are composited, and the image coding apparatus includes a transparency coding section for coding the transparency signal in addition to the shape coding section 1200 and the color coding section 1100, although this is not illustrated herein. The construction of the transparency coding section is almost identical to that of the color coding section 1100 and differs from the same in that the transparency signal is subjected to a motion compensation process according to the color motion vector MVt. That is, the transparency coding section includes no motion detector.
Subsequently, a description will be given of operation of such a constructed image coding apparatus 1000.
When the shape signal Ss of the image signal corresponding to a given object is input to the input terminal 1201 of the shape coding section 1200, the blocking unit 1210 divides the shape signal Ss into signals corresponding to respective macroblocks into which the rectangular region (bounding box) is divided, to generate blocked shape signals Ss. The blocked shape signal Bs is supplied to the shape motion detector 1240, the shape encoder 1220, and the mode decision unit 1280 in the shape coding unit 1200, the mode decision unit 1180 in the color coding unit 1100.
The shape motion detector 1240 detects the shape motion vector MVs of the target macroblock based on the blocked shape signal Bs and the reference shape signal MLds of the previous frame stored in the memory 1260. The shape motion vector MVs is output to the shape motion compensator 1250 and the motion vector coding unit 1270.
At this time, the shape motion detector 1240 also outputs the motion compensation error signal DFs of the shape signal which has been generated in the motion detection process to the mode decision unit 1280. The mode decision unit 1280 performs the coding mode decision process for the target macroblock based on the blocked image signal Bs and the motion compensation error signal DFs and outputs the shape coding mode signal Mos.
The shape motion compensator 1250 performs motion compensation to the blocked shape signal Bs based on the shape motion vector MVs and the reference shape signal MLds, to generate the prediction shape signal Ps for the target macroblock. The shape motion vector coding unit 1270 performs coding to the shape motion vector MVs according to the shape coding mode signal Mos and the information Mmvt and Mmot relating to the motion vectors from the motion vector coding unit 1170, and outputs the coded shape motion vector signal Emvs through the output terminal 1203.
The shape encoder 1220 performs arithmetic coding to the blocked shape signal Bs with or without reference to the prediction shape signal Ps according to the shape coding mode signal Mos, to generate the coded shape signal Cs and performs arithmetic decoding to the coded shape signal Cs to generate the locally decoded shape signal Lds, which is stored in the memory 1260 as the reference shape signal MLds.
The variable length encoder 1230 transforms the coded shape signal Cs and the shape coding mode signal Mos into variable length codes and outputs the resulting coded shape signal Es through the output terminal 1202.
When the color signal St of the image signal corresponding to the given object is input to the input terminal 1101 of the color coding section 1100, the blocking unit 1110 divides the color signal St into signals corresponding to respective macroblocks into which the rectangular region (bounding box) is divided, to generate blocked color signals St. The blocked color signal Bt is supplied to the color motion detector 1140, the color encoder 1120, and the mode decision unit 1180.
The color motion detector 1140 detects the color motion vector MVt of the target macroblock based on the blocked color signal Bt, the reference color signal MLdt stored in the memory 1160, and the locally decoded shape signal Lds from the shape coding section 1200. The color motion vector MVt is output to the color motion compensator 1150 and the motion vector coding unit 1170.
At this time, the color motion detector 1140 also outputs the motion compensation error signal DFt of the color signal generated in the motion detection process to the mode decision unit 1180. The motion decision unit 1180 performs the coding mode decision process for the target macroblock based on the blocked image signal Bt, the motion compensation error signal DFt, and the blocked shape signal Bs, and outputs the color coding mode signal Mot.
The color motion compensator 1150 performs motion compensation for the blocked color signal Bt based on the color motion vector MVt, the reference color signal MLdt, and the locally decoded shape signal Lds, to generate the prediction color signal Pt for the target macroblock. The color motion vector coding unit 1170 performs coding to the color motion vector MVt according to the color coding mode signal Mot. Thereby, the coded motion vector signal Emvt is output through the output terminal 1103 and the information Mmvt and Mmot relating to the color motion vector MVt is supplied to the motion vector coding unit 1270 in the shape coding unit 1200 from the color motion vector coding unit 1170.
The color encoder 1120 performs wave coding to the blocked color signal Bt with or without reference to the prediction color signal Pt based on the color coding mode signal Mot and the locally decoded shape signal Lds, to generate the coded color signal Ct and performs wave decoding to the coded color signal Ct to generate the locally decoded shape signal Ldt, which is stored in the memory 1160 as the reference color signal MLdt.
The variable length encoder 1130 transforms the coded color signal ct and the color coding mode signal Mot into variable length codes and outputs the coded color signal Et through the output terminal 1102.
A description will be given of an MPEG4 compliant image decoding apparatus.
Turning to FIG. 10, there is shown an image decoding apparatus 2000. The image decoding apparatus 2000 is adapted to perform a decoding process for the coded image signal sent from the image coding apparatus 1000 shown in FIG. 9, and comprises a color decoding section 2100 which performs a decoding process including a motion compensation decoding process for the coded color signal Et sent from the color coding section 1100 and a shape decoding section 2200 which performs a decoding process including the motion compensation decoding process for the coded shape signal Es sent from the shape coding section 1200.
The shape decoding section 2200 includes a variable length decoder 2210 which performs variable length decoding to the coded shape signal Es corresponding to an object input through an input terminal 2201 and outputs a decoded shape signal As and the shape coding mode signal Mos and a shape decoder 2220 which performs arithmetic decoding to the decoded shape signal As corresponding to the target macroblock with or without reference to a prediction shape signal Ps according to the coding mode signal Mos and outputs a decoded shape signal Ds.
The shape decoding section 2200 further includes a memory 2260 which stores the decoded shape signal Ds as a reference shape signal MDs of a decoded frame (previous frame), and an inverse blocking unit 2230 which integrates the decoded shape signals Ds in such a way that they correspond to respective macroblocks composing the bounding box and outputs the resulting regenerated shape signal Rs through an output terminal 2203.
The shape decoding section 2200 still further includes a motion vector decoding unit 2240 which decodes the coded shape motion vector signal Emvs sent from the image coding apparatus 1000 according to the shape coding mode signal Mos and the color motion vector information Mmvs and Mmot from the color decoding section 2100, to provide a decoded shape motion vector signal Dms, and a shape motion compensator 2250 which generates the prediction shape signal Ps for the target macroblock based on the decoded shape motion vector signal Dmvs and the reference shape signal MDs.
The color decoding section 2100 includes a variable length decoder 2110 which performs variable length decoding to the coded color signal Et corresponding to an object supplied through an input terminal 2101 and outputs a decoded color signal At and the color coding mode signal Mot, and a color decoder 2120 which performs wave coding to the decoded color signal At with or without reference to the prediction color signal Pt and outputs a decoded color signal Dt.
The color decoding section 2100 further includes a memory 2160 which stores the decoded color signal Dt as a reference color signal MDt of a decoded frame (previous frame), an inverse blocking unit 2130 which integrates the decoded color signals Dt in such a way that they correspond to respective macroblocks composing the bounding box and outputs the resulting regenerated color signal Rt through an output terminal 2103, a motion vector decoding unit 2140 which decodes the coded color motion vector signal Emvt sent from the image coding apparatus 1000 according to the color coding mode signal Mot, to provide a decoded color motion vector signal Dmvt, and outputs the information Mmvt and Mmot of the color motion vector, and a color motion compensator 2150 which generates a prediction color signal Pt for the target macroblock based on the decoded color motion vector signal Dmvt, the reference color signal MDt, and the decoded shape signal Ds from the shape decoding section 2200.
Subsequently, a description will be given of operation of such a constructed image decoding apparatus 2000.
When the coded shape signal Es corresponding to the object is input through the input terminal 2201, the variable length decoder 2210 performs variable length decoding to the coded shape signal Es in macroblock units to generate the decoded shape signal As and the shape coding mode signal Mos for each macroblock.
The coded shape motion vector signal Emvs is supplied to the motion vector decoding unit 2240 through the input terminal 2202, which decodes the signal Emvs according to the shape coding mode signal Mos and the information Mmvs and Mmot from the color decoding section 2100 to provide the decoded shape motion vector signal Dmvs. The shape motion compensator 2250 generates the prediction shape signal Ps for the target macroblock based on the decoded shape motion vector signal Dmvs and the reference shape signal MDs stored in the memory 2260.
The shape decoder 2220 performs arithmetic decoding to the decoded shape signal As for the target macroblock with or without reference to the prediction shape signal Ps to generate the decoded shape signal Ds, which is stored in the memory 2260 as the reference shape signal MDs, and output to the inverse blocking unit 2230, which integrates the decoded shape signals Ds in such a way that they correspond to respective macroblocks composing the bounding box and outputs the resulting regenerated shape signal Rs through the output terminal 2203.
When the coded color signal Et corresponding to the object is input through the input terminal 2101, the variable length decoder 2110 performs variable length decoding to the coded color signal Et in macroblock units, to generate the decoded color signal At and the color coding mode signal Mot for each macroblock.
The coded color motion vector signal Emvt is input to the motion vector decoding unit 2140 through the input terminal 2102, which decodes the signal Emvt according to the color coding mode signal Mot and outputs the decoded color motion vector signal Dmvt and the information Mmvt and Mmot relating to the color motion vector. The color motion compensator 2150 generates the prediction color signal Pt for the target macroblock based on the decoded color motion vector signal Dmvt, the reference shape signal MDt stored in the memory 2160, and the decoded shape signal Ds from the shape decoder 2220.
The color decoder 2120 performs wave decoding to the decoded color signal At corresponding to the target macroblock with or without reference to the prediction color signal Pt based on the color coding mode signal Mot and the decoded shape signal Ds from the shape decoding section 2000 to generate the decoded color signal Dt, which is stored in the memory 2160 as the reference color signal MDt and output to the inverse blocking unit 2130. The inverse blocking unit 2130 integrates the decoded color signals Dt in such a way that they correspond to respective macroblocks composing the bounding box and outputs the resulting regenerated color signal Rt through the output terminal 2103.
Hereinafter, a description will be given of a coding process for the color motion vector and the shape motion vector performed by the image coding apparatus 1000.
Initially, a description will be given of the motion compensation process by the use of the motion vector.
According to MPEG4, in a frame-by-frame motion compensation process for the color signal (interlaced or non-interlaced color signal) forming one macroblock (image space composed of (16.times.16) pixels, as shown in FIG. 11(a), four motion vectors MV1-MV4 are used to generate a prediction color signal for a target macroblock to-be-processed MB. The motion vectors MV1-MV4 correspond to four blocks (image spaces each composed of (8.times.8) pixels B1-B4 composing the macroblock MB, respectively. When these motion vectors MV1-MV4 are equivalent, the frame-by-frame motion compensation process for the macroblock MB is, as shown in FIG. 11(d), equivalent to the motion compensation process by the use of one motion vector MV (MV=MV1=MV2=MV3=MV4).
Meanwhile, in a field-by-field motion compensation process for the color signal (interlaced color signal) corresponding to one macroblock, as shown in FIG. 11(b), first and second motion vectors MVf1 and MVf2 are used to generate the prediction color signal for the target macroblock MB in a frame to-be-processed. The first motion vector MVf1 is a motion vector of a first sub-macroblock MBf1 comprising pixels on scanning lines in an odd field in the frame, and the second motion vector MVf2 is a motion vector of a second sub-macroblock MVf2 comprising pixels on scanning lines in an even field in the frame. Each of the sub-macroblocks is an image space comprising (16.times.8) pixels, although they are shown as square regions for convenience in the illustrated example shown in FIG. 11(b). When the first and second motion vectors MVf1 and MVf2 are equivalent, the field-by-field motion compensation process for the macroblock MB is, as shown in FIG. 11(d), equivalent to the frame-by-frame motion compensation process by the use of one motion vector MV (MV=MV1=MV2=MV3=MV4).
On the other hand, a frame-by-frame motion compensation process for the shape signal is different from the frame-by-frame motion compensation process for the color signal. As shown in FIG. 11(d), only the motion vector MV is used to generate the prediction shape signal for the target macroblock MB.
Subsequently, a description will be given of a process for coding motion vectors.
In the process for coding the motion vector (color motion vector) of the color signal corresponding to the target macroblock (16.times.16 pixels), difference (difference value) between a prediction motion vector predicted from a motion vector of a reference macroblock which has been coded and is positioned adjacently to the target macroblock and a motion vector of the target macroblock is coded as motion vector information of the target macroblock.
Hereinafter, a description will be given of the process for predicting the color motion vector with reference to FIGS. 12(a)-12(d).
(1) Motion Vector Prediction Process in Frame-by-Frame Motion Compensation Process
(1a) When a target macroblock MBx has four motion vectors, as shown in FIG. 12(a), the prediction value of a motion vector MVt0a of a block B0a in the target macroblock MBx is generated based on motion vectors MVt1a, MVt2a, and MVt2b of reference blocks RB1a, RB2a, and RB2b which are adjacent to the block B0a.
The prediction value of a motion vector MVt0b of a block B0b in the target macroblock MBx is, like the prediction value of the motion vector MVt0a, motion vectors MVt0a, MVt2b, and MVt3 of reference blocks B0a, RB2b, and RB3 which are adjacent to the block B0b. The prediction value of a motion vector MVt0c of a block B0c in the target macroblock MBx is, like the prediction value of the motion vector MVt0a, generated based on motion vectors MVt1b, MVt0a, and MVt0b of reference blocks RB1b, B0a, and B0b which are adjacent to the block B0c.
The prediction value of a motion vector MVt0d of a block B0d in the target macroblock MBx is generated based on motion vectors MVt0c, MVt0a, and MVt0b of reference blocks B0c, B0a, and B0b which are adjacent to the block B0d.
(1b) When a target macroblock MB0 has one motion vector MVt0, that is, when motion vectors MVt0a-MVt0d of respective blocks B0a-B0d in the target macroblock MBx are equivalent, as shown in FIG. 12(b), the prediction value of the motion vector MVt0 of the target macroblock MB0 is generated based on motion vectors MVt1, MVt2, and MVt3 of reference blocks RB1, RB2, and RB3 which are adjacent to the target macroblock.
In the above description of the motion vector prediction process in the frame-by-frame motion compensation process, the reference macroblock adjacent to the target macroblock in the macroblock which has been subjected to the frame-by-frame motion compensation process and has four motion vectors. In a case where the reference macroblock has one motion vector, based on the assumption that four blocks composing the reference macroblock have the same motion vector, the prediction values of respective blocks composing the macroblock are generated. In another case where the reference macroblock is a macroblock which has been subjected to the field-by-field motion compensation process and has two motion vectors, based on the average of these motion vectors, the prediction values of respective blocks are generated.
(2) Motion Vector Prediction Process in Field-By-Field Motion compensation Process
The field-by-field motion compensation process is, as shown in FIG. 12(c), for the case where a target macroblock MBy to-be-processed has first and second motion vectors MVt01 and MVt02. The first motion vector MVt01 is the motion vector of the sub-macroblock MBf1 comprising (16.times.8) pixels on scanning lines in the odd field in the frame, and the second motion vector MVt02 comprising (16.times.8) pixels is the motion vector of the sub-macroblock comprising pixels on scanning lines in the even field in the frame. In this case, the prediction values of the first and second motion vectors MVt01 and MVt02 of the target macroblock MBy are generated based on the motion vectors MVt1, MVt2, and MVt3 of the reference blocks RB1, RB2, and RB3 adjacent to the target macroblock MBy.
In the above description of the motion vector prediction process in the field-by-field motion compensation process, the reference macroblock adjacent to the target macroblock is the macroblock which has been subjected to the frame-by-frame motion compensation process and has four motion vectors. In a case where the reference macroblock has one motion vector, based on the assumption that four blocks composing the reference macroblock have the same motion vector, the prediction values of motion vectors MVt01 and MVt02 of the two sub-macroblocks MBf1 and MBf2 composing the target macroblock MBy are generated.
In another case where the reference macroblock is a macroblock which has been subjected to the field-by-field motion compensation process and has two motion vectors, the prediction value of the motion vector MVt01 of the sub-macroblock MBf1 corresponding to the odd field is generated based on the motion vector of the sub-macroblock corresponding to the odd field in the reference macroblock, and the prediction value of the motion vector MVt02 of the sub-macroblock MBf2 corresponding to the even field is generated based on the motion vector of the sub-macroblock corresponding to the even field in the reference macroblock.
Hereinafter, a description will be given of the shape motion vector prediction process in the shape signal motion compensation process. As described above, the target macroblock corresponding to the shape signal has always one motion vector.
In the motion vector (shape motion vector) coding process for the target macroblock (16.times.16 pixels), a difference value between the value of the prediction motion vector and the value of the shape motion vector of the target macroblock, i.e., the shape motion vector information of the target macroblock, is coded.
As already described, since there is high correlation between the motion vector of the shape signal and the motion vector of the color signal, the prediction value (prediction shape motion vector) of the shape motion vector of the target macroblock is generated based on both of the color motion vector of the coded reference macroblock and the shape motion vector of the coded reference macroblock.
The prediction value of the shape motion vector MVS0 of the target macroblock MBs0 shown in FIG. 12(d) is generated by referring to the color motion vectors MVt1, MVt2, and MVt3 of the blocks RB1, RB2, and RB3 adjacent to the target macroblock MB0 corresponding to the color signal shown in FIG. 12(b) as well as the vectors MVs1, MVs2, and MVs3 of the macroblocks RMBs1, RMBs2, and RMBs3 adjacent to the target macroblock MBs0 shown in FIG. 12(d).
In the process for predicting the color motion vector and the shape motion vector, if macroblocks adjacent to the target macroblock are intra-macroblocks or macroblocks outside the object, they have no motion vectors, in which case, the prediction value of the motion vector of the target macroblock is generated using adjacent macroblocks which have motion vectors as reference macroblocks.
FIG. 13 is a block diagram for explaining a prior art motion vector coding unit included in the image coding apparatus 1000. Shown in FIG. 13 is the motion vector coding unit (color motion vector coding unit ) 1170 included in the color coding section 1100 and the motion vector coding unit (shape motion vector coding unit) 1270 included in the shape coding section 1200. By way of example, a description will be given of a case where the target macroblock MB0 has one color motion vector and the blocks RB1-RB3 adjacent to the target macroblock MB0 are used as reference blocks (see FIG. 12(b)).
The color motion vector coding unit 1170 is adapted to perform coding to the motion vector of the non-interlaced color signal, and includes an MV memory 102 which stores the motion vector MVt of the target macroblock input through an input terminal 1 and an MV memory 103 which stores the color coding mode signal Mot input through an input terminal 2. The color coding mode signal Mot is the color motion vector effectiveness signal indicating that the target macroblock in the image space obtained from the color signal is one of the intra-macroblock and the macroblock outside the object or the inter-macroblock. According to this signal Mot, it is decided whether the reference macroblock adjacent to the target macroblock is one of the intra-macroblock and the macroblock outside the object, or the inter macroblock.
The color motion vector coding unit 1170 further includes an MV predictor 104 which generates an MV prediction value (the prediction value of the color motion vector of the target macroblock) Pmvt from the color motion vectors MVt1, MVt2, and MVt3 of the reference blocks RB1, RB2, and RB3 in the coded macroblock which have been stored in the MV memory 102 according to the color coding mode signal Mmot (Mot) from the MV memory 103, and an MV encoder 105 which encodes the color motion vector MVt0 of the target macroblock MB0 with reference to the MV prediction value Pmvt from the MV predictor 104 and outputs the coded color motion vector signal Emvt.
The color motion vector coding unit 1170 still further includes a primary switch 100 provided between the input terminal 1 and the MV encoder 105, and a secondary switch 101 provided between an output terminal 10 and the MV encoder 105, the switches 100 and 101 being on-off controlled by the color coding mode signal Mot.
The shape motion vector coding unit 1270 is adapted to perform coding to the motion vector of the non-interlaced shape signal and includes an MV memory 202 which stores the shape motion vector MVs of the target macroblock input through an input terminal 4, and an MV memory 203 which stores the shape coding mode signal Mos input through an input terminal 5. The shape coding mode signal Mos is the shape motion vector effectiveness signal indicating that the target macroblock in the image space obtained from the shape signal is one of the intra-macroblock and the macroblock outside the object or the inter-macroblock. According to this signal Mos, it is decided whether each of the reference macroblock adjacent to the target macroblock is one of the intra-macroblock and the macroblock outside the object, or the inter macroblock.
The shape motion vector coding unit 1270 further includes an MV predictor 204 which generates an MV prediction value (the prediction value of the shape motion vector for the target macroblock) Pmvs from the shape motion vectors MVs1, MVs2, and MVs3 in the coded macroblock which have been stored in the MV memory 202 according to the color coding mode signal Mot and the shape coding mode signal Mos, and an MV encoder 205 which encodes the shape motion vector MVS0 of the target macroblock MBs0 with reference to the MV prediction value Pmvs from the MV predictor 204 and outputs the coded shape motion vector signal Emvs. The MV predictor 204 is adapted to convert a 0.5 pixel-precision color motion vector from the MV memory 102 into a one pixel-precision color motion vector and perform the prediction process by the use of the one-pixel precision color motion vector. This is because the color coding section performs the motion compensation process by the use of the 0.5 pixel-precision while the shape coding section performs the motion compensation process by the use of the one-pixel precision.
The shape motion vector coding unit 1270 still further includes a primary switch 200 provided between an input terminal 4 and the MV encoder 205, and a secondary switch 201 provided between an output terminal 11 and the MV encoder 205, the switches 200 and 201 being on-off controlled by the shape coding mode signal Mos.
Subsequently, operation will be described.
The color motion vector coding unit 1170 performs coding to the color motion vector MVt and outputs the coded signal Emvt, and the shape motion vector coding unit 1270 performs coding to the shape motion vector MVs and outputs the coded signal Emvs.
When the color coding mode signal Mot is input to the input terminal 2 as the motion vector effectiveness signal, in accordance with this signal Mot, the switches 100 and 101 are on-off controlled. To be specific, when the color motion vector MVt0 of the target macroblock MB0 should be coded, the switches 100 and 101 are in the ON-state, or otherwise, they are in the OFF-state.
In other words, when the target macroblock is the intra-macroblock or the macroblock outside the object, and therefore the coding process for the motion vector is unnecessary, the switches 100 and 101 are in the OFF-state. On the other hand, when the target macroblock is the inter-macroblock, the switches 101 and 101 are in the ON-state.
The color motion vector MVt for each macroblock is temporarily stored in the MV memory 102. To be more detailed, the color motion vectors (see FIG. 11(a)) of the blocks composing each macroblock are stored in the MV memory 102, from which the motion vectors MVt1-MVt3 of the reference blocks MB1-MB3 adjacent to the target macroblock MB0 are supplied to the MV predictor 104. At this time, the signal Mot is temporarily stored in the MV memory 103.
The color motion vector effectiveness signal Hot for the target macroblock is also input to the MV predictor 104, which decides that each of the motion vectors MVt1-MVt3 should be coded according to the signal Mot, that is, each of the reference blocks RB1-RB3 is one of the intra-macroblock and the macroblock outside the object, or the inter-macroblock. According to the decision results, the prediction value (MV prediction value) Pmvt of the motion vector of the target macroblock MB0 is generated.
When decided that the reference blocks RB1-RB3 are the inter-macroblocks, the MV predictor 104 generates the prediction value of the motion vector of the target macroblock MB0 from the motion vectors Mvt1-Mvt3 of the coded reference blocks RB1-RB3 and outputs the prediction value Pmvt to the MV encoder 105. Note that the motion vectors of the reference blocks corresponding to the intra-macroblocks or the macroblocks outside the macroblock are not used for generating the prediction value.
The MV encoder 105 encodes the difference value (color difference motion vector) between the value of the motion vector MVt (MVt0) of the target macroblock and the prediction value Pmvt as the value of the color motion vector to generate the coded signal Emvt of the color motion vector, which is output through the secondary switch 101.
Likewise, when the shape coding mode signal Mos is input to an input terminal 5 as the shape motion vector effectiveness signal Mos, in accordance with this signal Mos, the switches 200 and 201 are on-off controlled. To be specific, when the color motion vector MVs0 of the target macroblock MBs0 is to be coded, the switches 200 and 201 are in the ON-state, or otherwise, they are in the OFF-state.
In other words, when the target macroblock is the intra-macroblock or the macroblock outside the object, and therefore the coding process for the corresponding motion vector is unnecessary, the switches 200 and 201 are in the OFF-state. On the other hand, when the target macroblock is the inter-macroblock, the switches 201 and 201 are in the ON-state.
The shape motion vector MVs for each macroblock is temporarily stored in the MV memory 202. To be more detailed, the motion vectors MVs1-MVs3 of the reference macroblock RMBs1-RMBs3 adjacent to the target macroblock MB0 are sent to the MV predictor 204 from the MV memory 102. The motion vectors MBt1-MBt3 of the blocks MBt1-MBt3 adjacent to the target macroblock MB0 shown in FIG. 12(c) are also sent to the MV predictor 204 from the MV memory 102. The signal Mmos is stored in the AV memory 203.
The MV predictor 204 is supplied with the shape motion vector signal Mmos of the target macroblock from the MV memory 203 and the color motion vector effectiveness signal Mmot (Mot) of the target macroblock from the MV memory 103. The MV predictor 204 decides whether or not the motion vectors MVs1-MVs3 of the reference macroblocks RMBs1-RMBs3 adjacent to the target macroblock should be coded, that is, decides whether or not each of the reference macroblocks is one of the intra-macroblock and the macroblock outside the object, or the inter-macroblock, according to the signal Mos. The MV predictor 204 decides whether or not the motion vectors MVt1-MVt3 of the reference macroblocks RB1-RB3 adjacent to the target macroblock should be coded, that is, decides whether or not each of the reference macroblocks RB1-RB3 is one of the intra-macroblock and the macroblock outside the object, or the inter-macroblock, according to the signal Mmot. According to the decision results, the prediction value (MV prediction value) Pmvs of the motion vector of the target macroblock MBs0 is generated.
When the reference macroblocks RMB1-RMB3 and the reference blocks RB1-RB3 are the inter-macroblocks, the MV predictor 204 generates the prediction value (MV prediction value) Pmvs of the motion vector of the target macroblock MBs0 from the motion vectors MVs1-MVs3 and the motion vectors MVt1-MVt3, and outputs the prediction value Pmvs to the MV encoder 205. Note that the motion vectors of the reference blocks corresponding to the intra-macroblocks or the macroblocks outside the object are not used for generating the prediction value.
The MV encoder 205 encodes the difference value (shape difference motion vector) between the value of the motion vector MVs of the target macroblock and the prediction value Pmvs as the value of the shape motion vector to generate the coded signal Emvs of the shape motion vector, which is output through the secondary switch 201.
FIG. 14 is a block diagram for explaining a prior art motion vector decoding unit included in the image decoding apparatus 2000. In FIG. 14, there is shown a motion vector decoding unit (color motion vector decoding unit) included in the color decoding section 2100 and a motion vector decoding unit (shape motion vector decoding unit) 2240 included in the shape decoding section 2200. Also in the description of the motion vector decoding unit, as in the case of the motion vector decoding unit, a description will be given of a case where the target macroblock MB0 has one color motion vector and the blocks RB1-RB3 of the target macroblock are used as reference blocks RB1-RB3 (see FIG. 12(b)).
The color motion vector decoding unit 2140 is adapted to decode the coded color motion vector Em output from the color motion vector coding unit 1170.
The color motion vector decoding unit 2140 includes an MV decoder 303 which decodes the coded color motion vector signal Emvt of the target macroblock to generate a decoded color motion vector signal Dmvt, and an MV memory 304 which temporarily stores the decoded color motion vector signal Dmvt. When the motion vector of the target macroblock MB0 is decoded, the motion vectors MVt1-MVt3 of the reference blocks RB1-RB3 (see FIG. 12(c)) are output from the MV memory 304.
The color motion vector decoding unit 2140 further includes an MV memory 302 which temporarily stores the color coding mode signal Mot for each macroblock input through an input terminal 6, and an MV predictor 305 which generates the MV prediction value Pmvt of the motion vector of the target macroblock from the motion vectors MVt1-MVt3 of the coded reference blocks RB1-RB3.
The color motion vector decoding unit 2140 still further includes a primary switch 300 provided between an input terminal 13 and an MV decoder 303, and a secondary switch 301 provided between an output terminal 20 and an MV decoder 303,the switches 300 and 301 being on-off controlled by the coding mode signal Mot.
The shape motion vector decoding unit 2240 is adapted to decode the coded shape motion vector Emvs output from the motion vector coding unit 1270 shown in FIG. 13.
The shape motion vector decoding unit 2240 includes an MV decoder 403 which decodes the coded shape motion vector signal Emvs of the target macroblock input to an input terminal 14 to generate a decoded shape motion vector signal Dmvs, and an MV memory 404 which temporarily stores the decoded shape motion vector signal Dmvs and outputs motion vectors MVs1-MVs3 of the reference macroblocks RMB1-RMB3 (see FIG. 12(d)).
The shape motion vector decoding unit 2240 further includes an MV memory 402 which stores the coding mode signal Mos of the target macroblock which is input through an input terminal 7, and an MV predictor 405 which generates the MV prediction value Pmvs of the motion vector MVs0 of the target macroblock MBs0 from the motion vectors MVs1-MVs3 of the reference macroblocks RMB1-RMB3 and the motion vectors MVt1-MVt3 of the reference blocks RB1-RB3 according to the coding mode signal Mmos and the coding mode signal Mot.
The shape motion vector decoding unit 2240 still further includes a primary switch provided between an input terminal 14 and an MV decoder 403 and a switch 401 provided between an output terminal 21 and an MV decoder 403, the switches 400 and 401 being on-off controlled by the coding mode signal MOS.
Subsequently, operation will be described.
The color motion vector decoding unit 2140 decodes the coded color motion vector signal Emvt sent from the color motion vector coding unit 1170 to generate the decoded color motion vector signal Dmvt. The shape motion vector decoding unit 2240 decodes the coded shape motion vector signal Emvs sent from the shape motion vector coding unit 1270 to generate the decoded shape motion vector signal Dmvs.
When the coding mode signal Mot decoded by the variable length decoder 2110 included in the color decoding section 2100 is input through the input terminal 6 as the motion vector effectiveness signal indicating whether or not the motion vector of the target macroblock is effective, the primary switch 300 and the secondary switch 301 is on-off controlled by the coding mode signal Mot. To be specific, when the coded color motion vector signal of the target macroblock is to be decoded, the switches 300 and 301 are in the ON-state, or otherwise the switches 300 and 301 are in the OFF-state.
In other words, when the target macroblock is the macroblock which has been intra-frame coded or the macroblock outside the object, and therefore the coding process for its motion vector is unnecessary, the switches are in the OFF-state, while the target macroblock is the macroblock which has been subjected to the motion compensation coding process (inter-frame coding process), the switches are in the ON-state.
The MV decoder 303 decodes the coded color motion vector signal Emvt with reference to the prediction value Pmvt of the color motion vector to generate the decoded color motion vector Dmvt, which is temporarily stored in the Mv memory 304.
When the color motion vector MVt0 of the target macroblock MB0 is decoded, the decoded signals Dmv of the motion vectors MVt1, MVt2, and MVt3 of the reference blocks RB1-RB3 shown in FIG. 12(c) are input to the MV predictor 305.
The coding mode signal Mot is temporarily stored in the MV memory 302. The MV predictor 305 decides whether or not each of the motion vectors MVt1-MVt3 has been coded according to the coding mode signal Mmot from the MV memory 302, that is, each of the reference blocks is either one of the intra-macroblock and the macroblock outside the object or the inter-macroblock. According to the decision results, the prediction value (MV prediction value) Pmvt of the motion vector MVt0 of the target macroblock MB0 is generated. The prediction value is generated as in the process performed by the color motion vector coding unit, and the resulting MV prediction value Pmvt is output to the MV decoder 303.
When the coding mode signal Mos decoded by the variable length decoder 2210 included in the shape decoding section 2200 is input through the input terminal 7, the switches 400 and 401 are on-off controlled. To be specific, when the coded shape motion vector signal of the target macroblock should be decoded, the switches 400 and 401 are in the ON-state, or otherwise the switches 400 and 401 are in the OFF-state.
In other words, when the target macroblock is the macroblock which has been intra-frame coded or the macroblock outside the object, and therefore the coding process for its motion vector is unnecessary, the switches 400 and 401 are in the OFF-state, while the target macroblock is the macroblock which has been subjected to the motion compensation coding process (inter-frame coding process), the switches are in the ON-state.
The MV decoder 403 decodes the coded shape motion vector signal Emvs with reference to the prediction value Pmvs to generate the decoded shape motion vector signal Dmvs, which is temporarily stored in the MV memory 404.
When the shape motion vector MVs0 of the target macroblock MBs0 is coded, the decoded signals Dmvs of the motion vectors MVs1, MVs2, and MVs3 of the reference macroblocks RMB1, RMB2, and RMB3 shown in FIG. 12(d) and the decoded signals Dmvt of the motion vectors MVt1, MVt2, and MVt3 of the reference blocks RB1, RB2, and RB3 shown in FIG. 12(c) are input to the MV predictor 405.
The coding mode signal Mos is temporarily stored in the MV memory 402. The MV predictor 405 is supplied with the signal Mos from the MV memory 402 and the signal Mmot from the MV memory 302.
The MV predictor 405 decides whether or not each of the shape motion vectors MVs1-MVs3 of the reference macroblocks, that is, decides whether or not each of them is one of the intra-macroblock and the macroblock outside the object, or the inter-macroblock, according to the signal Mmos. Also, the MV predictor 405 decides whether or not each of the motion vectors MVt1-MVt3 of the reference blocks, that is, decides whether or not each of the reference blocks is one of the intra-macroblock and the macroblock outside the object, or the inter-macroblock, according to the coding mode signal Mmot. According to the decision results, the prediction value Pmvs of the target macroblock Mbs0 is generated. The prediction value is generated as in the process performed by the color motion vector coding unit. The resulting MV prediction value Pmvs is output to the MV decoder 403.
In the image coding process according to MPEG4, an image signal to be processed can be extended to an interlaced image signal, like MPEG2. In this image coding process, field-by-field motion compensation can be performed to a color signal.
The field-by-field motion compensation according to MPEG4 is identical to that according to MPEG2. In the motion compensation process, as shown in FIG. 12(c), motion vectors MVt01 and MVt02 corresponding to blocks MBf1 and MBf2 in the first and second fields composing a macroblock are assigned to each macroblock in one image (one frame). That is, two motion vectors exist for each macroblock.
The motion compensation process according to MPEG4 employs a vector coding method in which a difference motion vector between the motion vector of a target macroblock and a motion vector predicted from the motion vector of a macroblock adjacent to the target macroblock is encoded as information of the motion vector of the target macroblock, thereby providing higher coding efficiency than that of the motion vector coding based on MPEG2.
FIG. 15 is a block diagram illustrating a prior art motion vector coding unit for an interlaced color signal.
The motion vector coding unit 1170a includes a frame MV memory 102a which stores motion vectors used for the frame-by-frame motion compensation (hereinafter referred to as frame-basis motion vectors); a field MV memory 102b which stores motion vectors used for the field-by-field motion compensation (hereinafter referred to as field-basis motion vectors); a field MV converter 110a which converts a frame-basis motion vector to field-basis motion vectors; and a frame MV converter 110b which converts field-basis motion vectors to a frame-basis motion vector.
Further, the motion vector coding unit 1170a includes an MV memory 103 which stores a coding mode signal Mot indicating whether the motion vector of a target macroblock is effective or not; a frame MV predictor 104a which generates a prediction value of the frame-basis motion vector from the frame-basis motion vector of a reference macroblock which has already been coded, according to the stored coding mode signal Mmot; and a field MV predictor 104b which generates a prediction value of the field-basis motion vector from the field-basis motion vector of a reference macroblock which has already been coded, according to the stored coding mode signal Mmot.
The frame MV predictor 104a refers to the motion vector stored in the frame MV memory 102a when the motion vector of the already coded reference macroblock is a frame-basis motion vector, and refers to a frame-basis motion vector which has been obtained by converting the field-basis motion vectors stored in the field MV memory 102b by the frame MV converter 110b when the motion vector of the reference macroblock is a field-basis motion vector. The field MV predictor 104b refers to the motion vector stored in the field MV memory 102b when the motion vector of the already coded reference macroblock is a field-basis motion vector, and refers to field-basis motion vectors obtained by converting the frame-basis motion vector stored in the frame MV memory 102a by the field MV converter 110a when the motion vector of the already coded reference block is a frame-basis motion vector.
Moreover, the motion vector coding unit 1170a includes a frame MV encoder 105a which encodes the frame-basis motion vector by referring to the MV prediction value output from the MV predictor 104a; and a field MV encoder 105b which encodes the field-basis motion vector by referring to the MV prediction value output from the MV predictor 104b.
Further, the motion vector coding unit 1170a includes a primary switch 100 having an input node S1i connected to the input terminal 1, a first output node S1a connected to the inputs of the frame MV memory 102a and the frame MV encoder 105a, a second output node S1b connected to the inputs of the field MV memory 102b and the field MV encoder 105b, and a third output node S1c which is open. The primary switch 100 connects the input node S1i to any of the first to third output nodes, according to the color coding mode signal Mot. The motion vector coding unit 1170a further includes a secondary switch 101 having an output node s2o connected to the output terminal 10, a first input node S2a connected to the output of the field MV encoder 105a, a second input node S2b connected to the output of the field MV encoder 105b, and a third input node S2c which is open. The secondary switch 101 connects the output node S2o to any of the first to third input nodes, according to the color coding mode signal Mot.
A description is given of the operation.
The color motion vector Mvt of the target macroblock is input to the input terminal 1, while the color motion vector effectiveness signal Mot of the target macroblock is input to the input terminal 2. The primary switch 100 and the secondary switch 101 are controlled according to the color motion vector effectiveness signal Mot.
That is, when the color motion vector MVt does not need to be coded, the input node S1i of the primary switch 100 is connected to the open output node S1c, and the output node S2o of the secondary switch 101 is connected to the open input node S2c.
When the color motion vector MVt is a frame-basis motion vector which needs to be coded, the input node S1i of the primary switch 100 is connected to the first output node S1a, and the output node S2o of the secondary switch 101 is connected to the first input node S2a.
Then, the motion vector MVt of the target macroblock, which has been input through the input terminal 1, is temporarily stored in the MV memory 102a. Further, in the field MV converter 110a, the frame-basis motion vector stored in the MV memory 102a is converted to the field-basis motion vectors.
When the color motion vector MVt is a field-basis motion vector which needs to be coded, the input node S1i of the primary switch 100 is connected to the second output node S1b, while the output node S2o of the secondary switch 101 connected to the second input node S2b.
Then, the motion vector MVt of the target macroblock, which has been input through the input terminal 1, is temporarily stored in the MV memory 102b. Further, in the frame MV converter 110b, the field-basis motion vectors stored in the MV memory 102b are converted to the frame-basis motion vector.
When the motion vector MVt of the target macroblock is a frame-basis motion vector, a prediction value of the frame-basis motion vector is generated in the frame MV predictor 104a, and the motion vector MVt of the target macroblock is encoded in the frame MV encoder 105a with reference to the prediction value to generate a coded signal Emvt1 of the frame-basis motion vector. The coded signal Emvt1 is output as a coded motion vector Emvt through the secondary switch 101.
Meanwhile, in the frame MV predictor 104a, when the reference macroblock is a macroblock subjected to the frame-by-frame motion compensation, the motion vector stored in the frame MV memory 102a is referred to. When the reference macroblock is a macroblock subjected to the field-by-field motion compensation, a frame-basis motion vector obtained by converting the field-basis motion vectors stored in the field MV memory 102b by the frame MV converter 110b is referred to.
On the other hand, when the motion vector MVt of the target macroblock is a field-basis motion vector, a prediction value of the field-basis motion vector is generated in the field MV predictor 104b, and the motion vector MVt of the target macroblock is encoded in the field MV encoder 105b with reference to the prediction value to generate a coded signal Emvt2 of the field-basis motion vector. This coded signal Emvt2 is output as a coded motion vector signal Emvt through the secondary switch 101.
In the field MV predictor 104b, when the reference macroblock is a macroblock subjected to the field-by-field motion compensation, the motion vector stored in the field MV memory 102b is referred to. When the reference macroblock is a macroblock subjected to the frame-by-frame motion compensation, field-basis motion vectors obtained by converting the frame-basis motion vector stored in the frame MV memory 102b by the field MV converter 110a is referred to.
FIG. 16 is a block diagram illustrating a prior art motion vector decoding unit for an interlaced color signal.
This motion vector decoding unit 2140a decodes the coded motion vector signal Emvt output from the motion vector coding unit 1170a shown in FIG. 15 to generate a decoded motion vector signal Dmvt.
The motion vector decoding unit 2140a includes a frame decoder 303a which decodes the coded signal of the frame-basis motion vector of the target macroblock by referring to the prediction motion vector obtained from the motion vector of an already decoded reference macroblock, to generate a decoded signal Dmvt1 of the frame-basis motion vector; and a field decoder 303b which decodes the coded signal of the field-basis motion vector of the target macroblock by referring to the prediction motion vector obtained from the motion vector of an already decoded reference macroblock, to generate a decoded signal Dmvt2 of the field-basis motion vector.
The motion vector decoding unit 2140a further includes a frame MV memory 304a which stores the decoded signal Dmvt1 of the frame-basis motion vector; a field MV memory 304b which stores the decoded signal Dmvt2 of the field-basis motion vector; a field MV converter 310a which converts the decoded signal of the frame-basis motion vector to the decoded signal of the field-basis motion vectors; and a frame MV converter 310b which converts the decoded signal of the field-basis motion vectors to the decoded signal of the frame-basis motion vector.
The motion vector decoding unit 2140a further includes an MV memory which stores a coding mode signal Mot indicating whether the motion vector of the target macroblock is effective or not; a frame MV predictor 305a which generates prediction value Pmvt1 of the frame-basis motion vector from a frame-basis motion vector which has already been decoded, according to the stored coding mode signal Mmot; and a field MV predictor 305b which generates a prediction value Pmvt2 of the field-basis motion vector from a field-basis motion vector which has already been decoded, according to the stored coding mode signal Mot.
The frame MV predictor 305a refers to the motion vector stored in the frame MV memory 304a when the motion vector of the already decoded reference macroblock is a frame-basis motion vector When the motion vector of the already decoded reference macroblock is a field-basis motion vector, the frame MV predictor 305a refers to a frame-basis motion vector which is obtained by converting the field-basis motion vectors stored in the field MV memory 304b by the frame MV converter 310b. The field MV predictor 305b refers to the motion vector stored in the field MV memory 304b when the motion vector of the already decoded reference macroblock is a field-basis motion vector. When the motion vector of the already decoded reference macroblock is a frame-basis motion vector, the field MV predictor 305b refers to the field-basis motion vectors obtained by converting the frame-basis motion vector stored in the frame MV memory 304a by the field MV converter 310a.
Further, the motion vector decoding unit 2140a includes a primary switch 300 having an input node S3i connected to the input terminal 13, a first output node 53a connected to the input of the frame MV decoder 303a, a second output node S3b connected to the input of the field MV decoder 303b, and a third output node S3c which is open. The primary switch 300 connects the input node S3i to any of the first to third output nodes, according to the coding mode signal Mot. The motion vector decoding unit 2140a further includes a secondary switch 301 having an output node S4o connected to the output terminal 20, a first input node 54a connected to the output of the frame MV decoder 303a, a second input node S4b connected to the output of the field MV decoder 303b, and a third input node S4c which is open. The secondary switch 301 connects the output node S4o to any of the first to third input nodes, according to the coding mode signal Mot.
A description is given of the operation.
The coded signal Emvt of the motion vector of the target macroblock is input to the input terminal 13, and the motion vector effectiveness signal Mot of the target macroblock is input to the input terminal 6. The primary switch 300 and the secondary switch 301 are controlled according to the motion vector effectiveness signal Mot.
Further, when the motion vector effectiveness signal Mot is stored in the MV memory 302, it is decided whether the motion vectors MVt1, MVt2 and MVt3 of the macroblocks to be referred to by the MV predictors 305a and 305b are encoded or not, and whether these vectors are frame-basis motion vectors or field-basis motion vectors, by the frame MV predictor 305a and the field MV predictor 305b, according to the effectiveness signal Mmot stored in the MV memory 302.
When the coded signal Emvt of the motion vector does not need to be decoded, the input node S3i of the primary switch 300 is connected to the open output node S3c, and the output node S4o of the secondary switch 301 is connected to the open input node S4c.
On the other hand, when the coded signal Emvt of the motion vector of the target macroblock is a frame-basis motion vector which needs to be decoded, the input node S3i of the primary switch 300 is connected to the first output node S3a, and the output node S4o of the secondary switch 301 is connected to the first input node S4a.
Then, the coded signal Emvt of the motion vector of the target macroblock, which has been input through the input terminal 13, is decoded in the frame MV decoder 303a with reference to the prediction motion vector Pmvt1 of the target macroblock to generate a decoded signal Dmvt1 of the motion vector. This decoded signal Dmvt1 is temporarily stored in the MV memory 304a. Further, in the field MV converter 310a, the frame-basis motion vector stored in the MV memory 304a is converted to the field-basis motion vectors.
Further, the frame MV predictor 305a performs frame-by-frame motion compensation in which a prediction motion vector of the motion vector of the target macroblock is generated by referring to the motion vector of an already decoded reference macroblock adjacent to the target macroblock. When the reference macroblock is a macroblock subjected to the frame-by-frame motion compensation, the motion vector stored in the frame MV memory 304a is referred to. When the reference macroblock is a macroblock subjected to the field-by-field motion compensation, a frame-basis motion vector obtained by converting the field-basis motion vectors stored in the field MV memory 304b by the frame MV converter 310b is referred to.
Next, when the coded signal Emvt of the motion vector of the target macroblock is a field-basis motion vector which needs to be decoded, the input node S3i of the primary switch 300 is connected to the second output node S3b, and the output node S4o of the secondary switch 301 is connected to the second input node S4b.
Then, the coded signal Emvt of the motion vector of the target macroblock, which has been input through the input terminal 13, is decoded by the field MV decoder 303b with reference to the prediction motion vector Pmvt2 of the target macroblock to generate a decoded signal Dmvt2 of the motion vector. The decoded signal Dmvt2 is temporarily stored in the MV memory 304b. Further, in the field MV converter 310b, the frame-basis motion vector stored in the MV memory 304b is converted to the field-basis motion vectors.
On the other hand, the field MV predictor 305b performs field-by-field motion compensation in which a prediction motion vector of the motion vector of the target macroblock is generated with reference to the motion vector of an already decoded reference macroblock adjacent to the target macroblock, according to the motion vector effectiveness signal of the target macroblock. In this process, when the reference macroblock is a macroblock subjected to the field-by-field motion compensation, the motion vector stored in the field MV memory 304b is referred to. When the reference macroblock is a macroblock subjected to the frame-by-frame motion compensation, a frame-basis motion vector obtained by converting the field-basis motion vectors stored in the frame MV memory by the frame MV converter 310b is referred to.
By the way, an interlaced signal used for ordinary TV broadcasting and a non-interlaced color signal or shape signal used for internet or data base are expected to be widely utilized in various fields in the future. However, it is thought that the applicable field of an interlaced shape signal would not be so wide.
As for "Next Generation TV" which has been much talked about in recent years, there is a high possibility of adopting non-interlacing and, therefore, from the practical point of view, it is not desired that the structures of coding and decoding apparatuses are complicated due to coding of an interlaced shape signal whose applicable field is not so wide.
Therefore, in order to realize coding and decoding of an interlaced shape signal without complicating the structures of coding and decoding apparatuses, the circuit structure for coding or decoding an interlaced color signal may be combined with the circuit structure for coding or decoding a non-interlaced shape signal.
So, in order to realize coding and decoding of an interlaced shape signal without complicating the structures of coding and decoding apparatuses, the circuit structure for coding or decoding an interlaced color signal may be combined with the circuit structure for coding or decoding a non-interlaced shape signal.
However, if non-interlace coding is performed to an interlaced shape signal, motion compensation of the interlaced shape signal would be performed frame by frame in accordance with the non-interlacing. In this case, when the color signal is an interlaced signal, there are two kinds of motion vectors relating to the color signal, i.e., a field-basis motion vector and a frame-basis motion vector. Therefore, it is difficult to perform coding of the shape motion vector with reference to the color motion vector as in the prior art motion vector coding.